This application generally relates to image processing, and more particularly, correcting frame-to-frame image changes due to platform motion for persistent observations.
There is a desire to collect persistent video (i.e., multiple image sequences) of a target from overhead platform-based (e.g., airborne or space-based) sensors that can easily be viewed, and/or interpreted, via displays. This may be especially important for military personnel and/or for other persons, using portable devices that may have limited processing capabilities. Conventional persistent video sensors generally stay fixed to (or focus on) a single point, for instance, on the ground, while the overhead platform is in motion. The motion of the platform, however, causes changes in scale, perspective (e.g. parallax), rotation, and/or other changes in viewing geometry. These changes can complicate or prevent human and/or machine interpretation of targets, features, and threats.
U.S. Pat. No. 8,471,915, issued Jun. 25, 2013, entitled “Self-Correcting Adaptive Long-Stare Electro-Optical System” (“SCALES”) and herein incorporated by reference in its entirety, discloses a system that is configured to calculate transformations to prevent image intra-frame distortion caused by a relative motion between the scene and the imaging platform and to prevent geometric differences from manifesting as smear within an integration time, thus preventing intra-frame distortion. However, this system relies upon controlling an optical element based on the transformation to prevent the image distortion, and may require more computations for intra-frame motion prevention.
Most nighttime imaging is performed at lower spatial resolution than daylight imaging either because longer wavelengths or fast optical systems are used to collect faint signals. Current short wave infrared (SWIR) staring sensors operate generally at video rates using very fast ˜F/1 optical systems. To collect higher resolution imagery with long focal length at a signal to noise ratio that enables interpretation, many frames of lowlight data must be aggregated to provide a high National Imagery Interpretability Rating Scales (NIIRS) image and a multi-spectral image. U.S. Pat. No. 9,294,755, issued Mar. 22, 2016, entitled “Correcting Frame-to-Frame Image Changes Due to Motion for Three Dimensional (3-D) Persistent Observations” (the '755 patent) and herein incorporated by reference in its entirety, demonstrated NIIRS improvement under nighttime low or no light imaging conditions, but required a priori knowledge of platform motion to develop coefficients to apply to eigenfunctions, each eigenfunctions describing a scene-wide transformation. These a priori computed coefficients were applied to multiple frames of imagery to remove induced motion effect. Those image correction methods ignored random errors (e.g., jitter) or errors in the prior knowledge.
An imaging platform having improved image quality thus is desired without the aforementioned drawbacks. For example, an imaging platform is desired which can correct frame-to-frame image changes caused by relative motion between the imaging platform and the scene. Further, an imaging platform is desired which can enhance the quality of captured images in low light applications that are particularly susceptible to inter-frame changes, e.g., imaging platforms having a wide field of view and/or high angular rates of movement with respect to the ground, especially for three-dimensional features in the scene.